Learning Humanoid Soccer Actions Interleaving Simulated and Real Data

This paper presents an approach for learning complex tasks on real robots, like walking or kicking in a humanoid soccer robot, profiting at most from the possibility to run simulations of a virtual model of the robot. This approach avoids to damage the real robot in the time consuming trials needed to learn a correct behavior and avoids to overfit the virtual robot model. The basic idea is to run most of the learning steps in simulation and to use a few learning steps on the real robot to assess discrepancies between the simulation and the reality. The calculated discrepancies are used to correct the fitness function used in simulation. Experiments on interleaving the learning between a real robot (Robovie-M by VStone) and its virtual model in USARSim are presented. They show that the proposed method is effective and significantly reduces learning time.